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DETAILED ACTION 

Claim Rejections - 35 USC § 103 

1 . The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

2. Claims 1, 4-7, 10-13, 18-25, and 28-34 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Murakami et al. (US 20050128291 A1, hereinafter referred 
to as "Murakami") in view of Koyanagi et al. (US 6720987 B2, hereinafter referred 
to as "Koyanagi") in view of Monroe (US 6970183 B1). 

Re claim 1, Murakami discloses a system, comprising: at least one video camera 
to capture warped panoramic video images of a scene and to produce a video stream 
(Murakami: Fig. 3, cameras C1 and C2); and a digital processor to receive and process 
said video stream (Murakami: Fig. 3, tracking processors 10 and 20), said digital 
processor comprising: a first processing module which provides pan, tilt, and zoom 
adjustments to allow for customized viewing of the scene (Murakami: Fig. 3, rotator 
drivers 1 1 and 21). 

Murakami does not specifically disclose that the first processing module unwarps 
warped panoramic video images to produce rectilinear video images of the scene. 
However, Koyanagi discloses a controller for a photographing apparatus and system, 
wherein a panoramic image is generated from multiple smaller component images 
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(Koyanagi: Figs. 4A-4F and 5A-5D) and a desired image may be expanded by driving a 
pan tilter device in response to user input (Koyanagi: column 3, lines 1-11). Since 
Murakami and Koyanagi relate to video monitoring systems, one of ordinary skill in the 
art at the time of the invention would have found it obvious to combine the pan tilting 
apparatus with panoramic image generation of Koyanagi with the system of Murakami 
in order to improve the ability of the user to acquire a desired image (Koyanagi: column 
2, lines 7-12). 

Murakami also discloses a second processing module for image processing 
(Murakami: Fig. 3, image processors 11 and 22-1), but neither Murakami nor Koyanagi 
explicitly disclose detecting and tracking a person's head in the rectilinear video images 
and to extract video images in the person's view from the rectilinear video images. 
However, Monroe indicates the capability of comparing face pictures with stored data 
for identification purposes (Monroe: column 23, lines 1-1 1) in addition to tracking 
capabilities (Monroe: column 28, line 62-column 29, line 2). Since Murakami, Koyanagi, 
and Monroe relate to surveillance and object tracking, one of ordinary skill in the art at 
the time of the invention would have found it obvious to combine the camera devices of 
Monroe with the system of Murakami and Koyanagi in order to improve response 
efficiency in the event of an alarm condition. 

Re claim 4, Murakami discloses that an intrusion alarm is activated in response 
to a moving object detected by the system (Murakami: paragraph [0051]). 
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Re claim 5, Murakami discloses a third processing module for discriminating 
moving objects (Murakami: Fig. 3, image processor 22-2), but Murakami does not 
explicitly disclose that said digital processing comprises a third processing module to 
process images of a face from streaming video images to for face recognition. 
However, Monroe indicates the capability of comparing face pictures with stored data 
for identification purposes (Monroe: column 23, lines 1-1 1) in addition to tracking 
capabilities (Monroe: column 28, line 62-column 29, line 2). Since Murakami, Koyanagi, 
and Monroe relate to surveillance and object tracking, one of ordinary skill in the art at 
the time of the invention would have found it obvious to combine the camera devices of 
Monroe with the system of Murakami and Koyanagi in order to improve response 
efficiency in the event of an alarm condition. 

Re claim 6, Murakami does not explicitly disclose that said third processing 
module performs a single-frame subspace feature analysis on the steaming video 
images to produce a sequence of classification results and a sequence of feature 
vectors and then processes said sequences separately to produce face recognition 
outputs. However, Monroe discloses the ability to look for patterns of activity (Monroe: 
column 34, lines 56-64). Since Murakami, Koyanagi, and Monroe relate to surveillance 
and object tracking, one of ordinary skill in the art at the time of the invention would 
have found it obvious to combine the camera devices of Monroe with the system of 
Murakami and Koyanagi in order to improve response efficiency in the event of an alarm 
condition. 
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Re claim 7, Murakami does not explicitly disclose that said third processing 
module is configured to apply a majority decision rule in processing the sequence of 
classification results. However, Monroe discloses that patterns and changes in patterns 
are determined, indicating that known patterns are used as a template, thus indicating 
that the closest pattern match is used as a basis for classification (Monroe: column 34, 
lines 56-64). Since Murakami, Koyanagi, and Monroe relate to surveillance and object 
tracking, one of ordinary skill in the art at the time of the invention would have found it 
obvious to combine the camera devices of Monroe with the system of Murakami and 
Koyanagi in order to improve response efficiency in the event of an alarm condition. 

Re claim 10, Murakami discloses a mechanism to measure a facial temperature 
pattern of a person (Murakami: Fig. 3, infrared camera C2). 

Re claim 11, Murakami does not specifically disclose a mechanism for 
performing a facial affect analysis on a person. However, Monroe indicates the 
capability of comparing face pictures with stored data for identification purposes 
(Monroe: column 23, lines 1-11) in addition to tracking capabilities (Monroe: column 28, 
line 62-column 29, line 2). Since Murakami, Koyanagi, and Monroe relate to 
surveillance and object tracking, one of ordinary skill in the art at the time of the 
invention would have found it obvious to combine the camera devices of Monroe with 
the system of Murakami and Koyanagi in order to improve response efficiency in the 
event of an alarm condition. 
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Re claim 12, Murakami does not specifically disclose a mechanism for 
performing a speech affect analysis. However, Monroe discloses that a microphone 
may be included in the camera appliance (Monroe: Figs. 8b, 8c, 8e, 8f, 8h, and 8i). 
Since Murakami, Koyanagi, and Monroe relate to surveillance and object tracking, one 
of ordinary skill in the art at the time of the invention would have found it obvious to 
combine the camera devices of Monroe with the system of Murakami and Koyanagi in 
order to improve response efficiency in the event of an alarm condition. 

Re claim 13, Murakami discloses a plurality of video cameras to capture video 
images of the scene at different locations, wherein said digital processor processes 
signals from said plurality of video cameras and said one video camera to detect and 
tract movement of an object in the scene (Murakami: Fig. 12). 

Murakami does not specifically disclose that the cameras capture warped 
panoramic video images. However, Koyanagi discloses a controller for a photographing 
apparatus and system, wherein a panoramic image is generated from multiple smaller 
component images (Koyanagi: Figs. 4A-4F and 5A-5D) and a desired image may be 
expanded by driving a pan tilter device in response to user input (Koyanagi: column 3, 
lines 1-11). Since Murakami and Koyanagi relate to video monitoring systems, one of 
ordinary skill in the art at the time of the invention would have found it obvious to 
combine the pan tilting apparatus with panoramic image generation of Koyanagi with 
the system of Murakami in order to improve the ability of the user to acquire a desired 
image (Koyanagi: column 2, lines 7-12). 
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Re claim 18, Murakami discloses a system, comprising: at least one video 
camera to capture warped panoramic video images of a scene and to produce a video 
stream (Murakami: Fig. 3, cameras C1 and C2); and a video transmission mechanism 
to deliver said video stream to a plurality of video receivers (Murakami: Fig. 4, network 
interface 46). 

Murakami discloses a picture recording device (Murakami: Fig. 5, picture 
recording device 53), but does not explicitly disclose said video transmission 
mechanism comprising a video server which filters information in said video stream 
according to security levels assigned to video receivers to produce different filtered 
video streams to different video receivers. However, Monroe discloses a multimedia 
surveillance system which includes filtering whom is contacted regarding an alarm event 
by on-duty status (Monroe: column 30, lines 41-48) and proximity to the alarm event 
location (Monroe: column 30, lines 48-54). Since both Murakami and Monroe relate to 
surveillance and object tracking, one of ordinary skill in the art at the time of the 
invention would have found it obvious to combine the notification filtering of Monroe with 
the system of Murakami in order to improve response efficiency in the event of an alarm 
condition. 

Murakami further discloses image processors 12 and 22-1 (in Murakami: Fig. 3), 
but neither Murakami nor Monroe explicitly discloses a digital processor in each video 
receiver to independently process said video stream to unwrap said warped panoramic 
video images to produce rectilinear video images of the scene, said digital processor 
having a user graphic interface with pan, tilt, and zoom adjustments to allow for 
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customized viewing at each video receiver. However, Koyanagi discloses a controller 
for a photographing apparatus and system, wherein a panoramic image is generated 
from multiple smaller component images (Koyanagi: Figs. 4A-4F and 5A-5D) and a 
desired image may be expanded by driving a pan tilter device in response to user input 
(Koyanagi: column 3, lines 1-11). Since Murakami, Monroe, and Koyanagi relate to 
video monitoring systems, one of ordinary skill in the art at the time of the invention 
would have found it obvious to combine the pan tilting apparatus with panoramic image 
generation of Koyanagi with the combined system of Murakami and Monroe in order to 
improve the ability of the user to acquire a desired image (Koyanagi: column 2, lines 7- 
12). 

Re claim 19, Murakami discloses that said video servo includes a digital process 
to tracks a change in the scene and adjusts filtering in a filtered video stream according 
to the change in the scene (Murakami: paragraph [0033]). 

Re claim 20, Murakami does not specifically disclose that one video receiver 
includes a PDA. However, Monroe discloses that the system may include a PDA, 
laptop, or other work station which includes a CPU (Monroe: column 16, line 58-column 
17, line 5). Since both Murakami and Monroe relate to surveillance and object tracking, 
one of ordinary skill in the art at the time of the invention would have found it obvious to 
combine the notification devices of Monroe with the system of Murakami in order to 
improve response efficiency in the event of an alarm condition. 
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Re claim 21, Murakami does not specifically disclose that one video receiver 
includes a laptop computer. However, Monroe discloses that the system may include a 
PDA, laptop, or other work station which includes a CPU (Monroe: column 16, line 58- 
column 17, line 5). Since both Murakami and Monroe relate to surveillance and object 
tracking, one of ordinary skill in the art at the time of the invention would have found it 
obvious to combine the notification devices of Monroe with the system of Murakami in 
order to improve response efficiency in the event of an alarm condition. 

Re claim 22, Murakami does not specifically disclose that one video receiver 
includes a desktop computer. However, Monroe discloses that the system may include 
a PDA, laptop, or other work station which includes a CPU (Monroe: column 16, line 58- 
column 17, line 5). Since both Murakami and Monroe relate to surveillance and object 
tracking, one of ordinary skill in the art at the time of the invention would have found it 
obvious to combine the notification devices of Monroe with the system of Murakami in 
order to improve response efficiency in the event of an alarm condition. 

Re claim 23, Murakami does not specifically disclose that said video camera is 
an omni-directional video camera to capture a 360-degree view of the scene. However, 
Monroe discloses that the cameras of the system have a full 360 degree field of view 
(Monroe: column 28, lines 52-57). Since both Murakami and Monroe relate to 
surveillance and object tracking, one of ordinary skill in the art at the time of the 
invention would have found it obvious to combine the camera devices of Monroe with 
the system of Murakami in order to improve response efficiency in the event of an alarm 
condition. 
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Re claim 24, Murakami does not explicitly disclose that said digital processor 
includes a video-based face recognition module which processes multiple images of a 
face from a video to perform face recognition. However, Monroe indicates the capability 
of comparing face pictures with stored data for identification purposes (Monroe: column 
23, lines 1-11). Since both Murakami and Monroe relate to surveillance and object 
tracking, one of ordinary skill in the art at the time of the invention would have found it 
obvious to combine the camera devices of Monroe with the system of Murakami in order 
to improve response efficiency in the event of an alarm condition. 

Re claim 25, Murakami does not explicitly disclose that said digital processor 
includes a module that detects and tracks a person's head. However, Monroe indicates 
the capability of comparing face pictures with stored data for identification purposes 
(Monroe: column 23, lines 1-11) in addition to tracking capabilities (Monroe: column 28, 
line 62-column 29, line 2). Since Murakami, Koyanagi, and Monroe relate to 
surveillance and object tracking, one of ordinary skill in the art at the time of the 
invention would have found it obvious to combine the camera devices of Monroe with 
the system of Murakami and Koyanagi in order to improve response efficiency in the 
event of an alarm condition. 

Re claim 28, Murakami does not explicitly state that said digital processor 
includes a tracking module to detect and track a location of an object or a person in real 
time. However, Monroe discloses that the sensing of unauthorized activity and 
activation of alarms occurs in real-time (Monroe: column 29, lines 36-44). Since 
Murakami, Koyanagi, and Monroe relate to surveillance and object tracking, one of 
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ordinary skill in the art at the time of the invention would have found it obvious to 
combine the camera devices of Monroe with the system of Murakami and Koyanagi in 
order to improve response efficiency in the event of an alarm condition. 

Re claim 29, Murakami does not explicitly disclose that said video transmission 
mechanism includes a wired communication link. However, Monroe discloses that both 
wired and wireless appliance and sensor systems may be employed (Monroe: Abstract 
section). Since Murakami, Koyanagi, and Monroe relate to surveillance and object 
tracking, one of ordinary skill in the art at the time of the invention would have found it 
obvious to combine the camera devices of Monroe with the system of Murakami and 
Koyanagi in order to improve response efficiency in the event of an alarm condition. 

Re claim 30, Murakami does not explicitly disclose that said video transmission 
mechanism includes a wireless communication link. However, Monroe discloses that 
both wired and wireless appliance and sensor systems may be employed (Monroe: 
Abstract section). Since Murakami, Koyanagi, and Monroe relate to surveillance and 
object tracking, one of ordinary skill in the art at the time of the invention would have 
found it obvious to combine the camera devices of Monroe with the system of Murakami 
and Koyanagi in order to improve response efficiency in the event of an alarm condition. 

Re claim 31, Murakami does not specifically disclose that said video 
transmission mechanism includes a video server that removes selected image 
information from a video signal to send a modified video signal to a video receiver. 
However, Monroe discloses that video may be transmitted in a sequential switching 
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method (Monroe: column 15, line 66-column 16, line 26), and upon an alarm event 
detection the switching may be modified to focus on the alarm event (Monroe: column 
16, lines 36-54). Since Murakami, Koyanagi, and Monroe relate to surveillance and 
object tracking, one of ordinary skill in the art at the time of the invention would have 
found it obvious to combine the camera devices of Monroe with the system of Murakami 
and Koyanagi in order to improve response efficiency in the event of an alarm condition. 

Re claim 32, arguments analogous to those presented for claim 1 are applicable 
to claim 32. Therefore, claim 32 has been analyzed and rejected with respect to the 
citations for claim 1 above. 

Re claim 33, arguments analogous to those presented for claim 18 are 
applicable to claim 33. Therefore, claim 33 has been analyzed and rejected with 
respect to the citations for claim 18 above. 

Re claim 34, Murakami does not specifically disclose that said digital image is a 
map of an area including the scene. However, Monroe discloses that events may be 
triangulated from multiple sensors and displayed on a map (Monroe: column 28, lines 8- 
12). Since Murakami, Koyanagi, and Monroe relate to surveillance and object tracking, 
one of ordinary skill in the art at the time of the invention would have found it obvious to 
combine the camera devices of Monroe with the system of Murakami and Koyanagi in 
order to improve response efficiency in the event of an alarm condition. 
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3. Claims 2, 8, and 9 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Murakami et al. (US 20050128291 A1, hereinafter referred to as "Murakami") 
in view of Koyanagi et al. (US 6720987 B2, hereinafter referred to as "Koyanagi") 
in view of Monroe (US 6970183 B1) as applied to claims 1, 4-7, 10-13, 18-25, and 
28-34 above, and further in view of Lin et al. (US 20030058340 A1, hereinafter 
referred to as "Lin"). 

Re claim 2, neither Murakami, Koyanagi, nor Monroe explicitly disclose that said 
second processing module performs an edge detection in the rectilinear video images to 
extract features and an ellipse detection to extract possible head images. However, Lin 
discloses a video monitoring system, wherein foreground objects are segmented from 
background regions (Lin: paragraph [0025]), including segmentation of a head (Lin: 
paragraph [0031]). This segmentation indicates that edges are found so that 
foreground/background regions may be identified, as is well known in the art. Since Lin 
relates to video monitoring, one of ordinary skill in the art at the time of the invention 
would have found it obvious to combine the segmentation of Lin with the system of 
Murakami, Monroe, and Koyanagi in order to identify complex time-varying behaviors 
for event classification. 

Re claim 8, neither Murakami, Monroe, nor Koyanagi specifically discloses that 
said third processing module is configured to apply a discrete hidden Markov model 
decision rule in processing the sequence of classification results. However, Lin 
discloses that event learning and classification may be accomplished using a 
hierarchical hidden Markov model (HMM) (Lin: paragraphs [0009]-[0010]). Since Lin 
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relates to video monitoring, one of ordinary skill in the art at the time of the invention 
would have found it obvious to combine the segmentation of Lin with the system of 
Murakami, Monroe, and Koyanagi in order to identify complex time-varying behaviors 
for event classification. 

Re claim 9, neither Murakami, Monroe, nor Koyanagi specifically discloses that 
said third processing module is configured to apply a continuous density hidden Markov 
model decision rule in processing the sequence of feature vectors. However, Lin 
discloses that event learning and classification may be accomplished using a 
hierarchical hidden Markov model (HMM) (Lin: paragraphs [0009]-[0010]). Since Lin 
relates to video monitoring, one of ordinary skill in the art at the time of the invention 
would have found it obvious to combine the segmentation of Lin with the system of 
Murakami, Monroe, and Koyanagi in order to identify complex time-varying behaviors 
for event classification. 

4. Claims 3, 14-17, 26, and 27 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Murakami et al. (US 20050128291 A1, hereinafter referred to as 
"Murakami") in view of Koyanagi et al. (US 6720987 B2, hereinafter referred to as 
"Koyanagi") in view of Monroe (US 6970183 B1) as applied to claims 1, 4-7, 10-13, 
18-25, and 28-34 above, and further in view of Murching et al. (US 6917692 B1, 
hereinafter referred to as "Murching"). 

Re claim 3, neither Murakami, Koyanagi, nor Monroe explicitly disclose that said 
second processing module performs Kalman filtering to predict an orientation of and 
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track motion of the person's head. However, Murching discloses Kalman tracking of 
color objects, wherein objects of interest are identified and their movements are 
predicted using a Kalman algorithm (Murching: column 1, lines 37-49). Since 
Murakami, Monroe, Koyanagi, and Murching relate to video monitoring, one of ordinary 
skill in the art at the time of the invention would have found it obvious to combine the 
Kalman tracking of Murching with the system of Murakami, Monroe, and Koyanagi in 
order to provide an accurate and robust tracking system that is resistant to error 
(Murching: column 1 , lines 46-49). 

Re claim 14, neither Murakami, Monroe, nor Koyanagi specifically disclose that 
said digital processor performs shadow detection from each video signal from each 
video camera to segment the object from the scene. However, Murching discloses that 
objects are separated by color (Murching: Fig. 7), thereby indicating that shadows 
(typically black or gray) may be segmented from color objects. Since Murakami, 
Monroe, Koyanagi, and Murching relate to video monitoring, one of ordinary skill in the 
art at the time of the invention would have found it obvious to combine the Kalman 
tracking of Murching with the system of Murakami, Monroe, and Koyanagi in order to 
provide an accurate and robust tracking system that is resistant to error (Murching: 
column 1, lines 46-49). 

Re claim 15, Murakami discloses that said digital processor performs a 
triangulation according to positions of the video cameras to produce horizontal positions 
x, y of the object and a height estimation algorithm to produce averaged vertical position 
z of the object (Murakami: Fig. 8 and paragraph [0034]). 
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Re claim 16, Murakami discloses that said digital processor registers a track of 
movement for the object according to x and y positions (Murakami: Fig. 8 and 
paragraph [0034]). 

Re claim 17, Murakami discloses that the triangulation is performed by using an 
extended N-ocular algorithm (Murakami: Fig. 8 shows 2 cameras used in the 
triangulation). 

Re claim 26, neither Murakami, Monroe, nor Koyanagi specifically discloses that 
said module further detects a face orientation of the person. However, Murching 
discloses Kalman tracking of color objects, wherein objects of interest are identified and 
their movements are predicted using a Kalman algorithm (Murching: column 1, lines 37- 
49). Since Murakami, Monroe, Koyanagi, and Murching relate to video monitoring, one 
of ordinary skill in the art at the time of the invention would have found it obvious to 
combine the Kalman tracking of Murching with the system of Murakami, Monroe, and 
Koyanagi in order to provide an accurate and robust tracking system that is resistant to 
error (Murching: column 1 , lines 46-49). 

Re claim 27, neither Murakami, Monroe, nor Koyanagi specifically discloses that 
said module further extracts a video image in the person's view from the video 
according to estimated face orientation. However, Murching discloses Kalman tracking 
of color objects, wherein objects of interest are identified and their movements are 
predicted using a Kalman algorithm (Murching: column 1, lines 37-49). Since 
Murakami, Monroe, Koyanagi, and Murching relate to video monitoring, one of ordinary 
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skill in the art at the time of the invention would have found it obvious to combine the 
Kalman tracking of Murching with the system of Murakami, Monroe, and Koyanagi in 
order to provide an accurate and robust tracking system that is resistant to error 
(Murching: column 1, lines 46-49). 

Conclusion 

5. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure: 

a. Loveland (US 643781 9 B1 ), Automated video person tracking system 

b. Smith et al. (US 5870138 A), Facial image processing 

c. Bauer et al. (US 20020176603 A1), Automatic pan/tilt pointing device, 
luminaire follow-spot, and 6DOF 3D position/orientation calculation information 

d. August (US 20030156762 A1), Volterra filters for enhancement of 
contours in images 

Contact 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to CHRISTOPHER FINDLEY whose telephone number is 
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